Tag: til

I went down a rabbit hole comparing four ways to turn unstructured documents into structured data: full LLM inference, fine-tuned small models, template-based extraction, and cloud OCR. The cost spread floored me. A template reads a document for about $0.001, where full LLM inference runs $5 to $15 on the same PDF and the same fields. Most teams pay LLM prices for forms a regex could handle. If you classify each document upfront and send it to the cheapest tool that can do the job, you cut costs by 85% and still keep the big model in reserve for the genuinely weird formats.

I went down a semantic layer rabbit hole recently, so here are the notes before they evaporate. It's the translation layer between your raw data and the people asking questions of it. Define "revenue" once and it means the same thing whether you reach it through SQL, a BI tool, an API, or one of your AI agents, which kills most of the governance drama before it starts. What really pulled me in was the risk angle. Every extract into a cube or a data mart leaves behind a stale copy that needs its own governance and gives you one more surface to breach, so I've started paying attention to the vendors that skip the copy and build aggregates in place, like AtScale on Databricks.

Today I learned you test LinkedIn sharing in production, because there's nowhere else to test it. LinkedIn's crawler scrapes your Open Graph tags once and caches them, so fixing a typo afterward just leaves a stale preview and never tells you it did. Clearing your own browser cache does nothing, because the cache lives on their servers, not yours. You can't test it locally either, since the crawler never reaches localhost. What did eventually get me unstuck was LinkedIn's Post Inspector, a ?v=2 cache-bust on the URL, and checking that my Django Site domain was actually right.

I used to picture multi-head attention as cutting the embedding into pieces, one slice per head. That's wrong. Every head reads the whole input embedding, all 768 dimensions of it, and projects it down through its own learned weights, so none of them works with a smaller input than the others. The weights are what pull each head in a different direction, and the model learns those during training rather than getting them from any fixed split of the embedding.

Flask's request context object is called g, and the name is too clever by half. It stands for global. Except each request gets its own isolated g, so you get global-like access without the race conditions a real shared global would hand you. Armin Ronacher chose the letter on purpose. It's quick to type when you reach for it constantly, and a single letter stands out on the page.

AWS DMS (Database Migration Service) gets overlooked as a CDC tool because the marketing is all about migrations. For simple change data capture it works well, and it barely needs configuring thanks to native Aurora Postgres support. I only pull in something heavier once I need exactly-once delivery, sub-second latency, or the like.

I kept seeing reranking in RAG discussions and finally clicked that it names a specific technique. Your fast initial retrieval grabs a batch of candidate documents. Then a slower, sharper model re-scores just those and reorders them before the LLM ever sees them. That second pass is where you get to weigh recency or source authority, the kind of scoring that would melt your servers if you ran it across the whole corpus.